Large software development projects can have multiple phases, including specification, development, and testing. Various software development methodologies include repeating some or all of these phases multiple times, such as in large or complex software development projects. Professional software development teams generally employ testers to test software before it is released to customers. The testers may test software to ensure correctness, completeness, security, and quality. When the tested software does not conform to a tester's expectations, the tester may identify a software defect (“bug”). The tester may provide a sequence of steps so that a software developer can reproduce the defect. The software developer may then resolve the defect, such as by fixing source code and producing a new “build” of the software. It is well known in the art that fixing bugs sometimes introduces other bugs. Thus, testers often perform regression testing, which could involve following the steps previously identified as producing the defect.
Various software testing techniques exist. These techniques can generally be classified as manual testing and automated testing. Manual testing requires a human to perform most of the testing steps. As an example, the human may test the software by either following various scenarios that detail a potential user's interactions with the software or by taking various steps, such as randomly, to identify defects. When performing automated testing, software testers use test automation tools to automatically cause the tested software to take various steps. For example, test automation tools can record a tester's interactions with software as steps and then play back the steps. Some test automation tools employ frameworks that interact with tested software programmatically, such as by using an application program interface (API) provided by the tested software.
Automated testing techniques other than recording and playing back steps can further be divided into keyword-based testing and model-based testing. In keyword-based testing, each discrete interaction with the tested software is assigned a keyword and can have associated parameters. As an example, the process of logging in can be associated with a keyword “login” and have as associated parameters a user identifier and a password. To automatically test software, the software tester may specify one or more keywords, such as in a sequence, so that the test automation tool performs steps relating to each specified keyword. In model-based testing, the software tester specifies (or causes to be specified) a state engine model of the tested software. The state engine model can identify a set of states relating to the tested software and interactions that cause the tested software to move from one state to another. A software tester can then specify a test suite that moves the tested software from, through, or to various states. As an example, the test suite may specify that the tested software is to progress from a “not logged in” state through a “logging in” state to a “logged in” state.
Conventionally, keyword-based testing was thought to be useful for regression testing. On the other hand, model-based testing was thought to be good for discovering software defects.
In the drawings, identical reference numbers identify similar elements or acts. The sizes and relative positions of elements in the drawings are not necessarily drawn to scale. For example, the shapes of various elements and angles are not drawn to scale, and some of these elements are arbitrarily enlarged and positioned to improve drawing legibility. Further, the particular shapes of the elements as drawn are not intended to convey any information regarding the actual shape of the particular elements, and have been solely selected for ease of recognition in the drawings.